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Video encoding with constrained fluctuations of quantizer scale ^ 

2 8 «* 2002 



The invention relates to a method of encoding video data and a system 
implementing such a method. 



5 An article titled "On performance gains in MPEG-2 video coding via a rate- 

distortion optimal route" by W.M.J. Coeiie and G.J, Keesman, published in "Signal 
Processing" 55 (1996) pp. 369-374, discloses methods of computing MPEG encoded video 
data. The MPEG video encoding standard provides for compression of video information. 
Video frames are coded as I (Intra) frames that are encoded by themselves or P and B frames 
10 that are encoded in terms of updates to other frames. Video frames are divided into blocks 
(MPEG distinguishes blocks and macroblocks, the latter comprising a number of blocks with 
information from different color channels, but since this does not affect the invention, the 
general term "block" will be used indiscriminately for blocks and macroblocks unless 
mentioned otherwise). Some blocks are made zero, a DCT (Discrete Cosine Transform) 
1 5 coefficients are computed of the video information in each remaining block, the coefficients 
are quantized and the quantized coefficients are encoded into an MPEG signal. 

Encoding by means of updates, zeroing blocks and quantizing are the main 
contributors to compression of video information in MPEG. Quantizing involves replacing 
the coefficients with quantized coefficients selected from a limited set of available values. 
20 The limited set is characterized by a quantizer scale value Q, which corresponds to the 

difference between successive available values. Replacement of the coefficients by quantized 
coefficients reduces the amount of information that is needed to encode the coefficients, be it 
at the expense of introducing distortion of the image by means of rounding errors due to the 
differences between the original coefficients and the quantized coefficients. In an MPEG 
25 signal the quantizer scale value Q may be varied from block to block, so as to select an 
optimal balance between compression and distortion of the encoded signal. 

The article by Coene and Keesman discloses an optimization method for 
selecting optimal compression parameters, including the quantizer scale values. A measure of 
distortion is minimized under the constraint that no more than a certain amount of 
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information (bit-rate) may be used to encode the video signal. The particular measure of 
distortion used in this method is not disclosed, but presumably an aggregate of distortions of 
individual frames is used. 

The article shows that in prior methods the distortion of I frames on one hand 
5 differed the distortion of B and P frames on the other hand. Thus, the distortion varied 

periodically with the so-called group of frame frequency (a group of frames containing an I 
frame and surrounding B and P frames). In one embodiment of the method of the article 
reduces this variation of distortion was reduced by optimizing an aggregate of the distortions 
of a plurality the frames in such a group. In another embodiment this variation was reduced 
0 by making the distortion of all frames equal. 

However, in the known method of encoding, human viewers still notice visual 
artifacts for some video signals when the encoded signal is decoded. For instance, at low 
encoding bit-rates a heartbeat effect may be observed due to alternating periods of better and 
worse coding of blocks. This heartbeat effect is most noticeable in difficult scenes, for 
instance, during zooming in on or out of a particular object, or at edges of relatively uniform 
surfaces, which are liable to have been encoded with fewer bits and are therefore prone to 
more errors. This effect may also be present in transcoded data streams and is then due to the 
buffer regulation algorithm. 



It is an objective of the invention to reduce the above mentioned heartbeat 

effect. 

The invention is based on the realization that the noticeable distortion due to 
quantization is not so much a matter of distortion of frames, i.e. difference between 
compressed and uncompressed images, but rather a matter of local variation of the distortion. 
This effect is not measured by normal aggregate measures of distortion. 

By using the encoding method according to the invention, the fluctuations of 
the quantizer scale of blocks at the same position in the frames as a function of time and/or in 
the same image as a function of position are dampened. In an embodiment of the invention, a 
spatial and/or temporal filter is applied over the pre-selected quantizer scale values, thereby 
limiting a change in the quantization scale as a function of time and/or position. 

The invention aims to locally constrain fluctuations about a trend around 
which the quantizer scale may fluctuate. The quantizer scale may follow the trend, but 
temporal or spatial fluctuations around the trend are constrained, so that the effect of 
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fluctuations in a local environment in the temporal or space domain is damped. The invention 
further relates to an encoder an audiovisual device, a data container device, a computer 
program and a data carrier device on which a computer program is stored. 

In an embodiment the method is applied to transcoding of a video stream that 
has already been quantized. Thus generation of artifacts during transcoding is minimized. 

In another embodiment estimated motion vectors are used to select blocks in 
different frames, that are related to each other by the motion vectors. The provisionally 
selected quantization values for the blocks that found to successively correspond with each 
other in successive frames are temporally low-pass filtered. 



Further objects, elaborations, modifications, effects, and details of the 
invention appear from the following description, in which reference is made to the drawing. 
Fig. 1 shows an encoding apparatus; 
1 5 Fig. 2 shows a flow diagram of an encoding method; 

Fig. 3 shows a graph of quantizer scale as a function of time ; 
Fig. 4 shows a transcoding apparatus. 



20 Figure 1 shows an encoding apparatus. The encoding apparatus has an input 

10 and an output 17 and the apparatus contains a pre-processor 12, a quantizer 14, a packager 
16, a quantizer scale selector 18 and a quantizer scale filter 19. The input is coupled to the 
output via, successively, pre-processor 12, quantizer 14 and packager 16. Quantizer scale 
selector 18 has an input coupled to input 10 and an output coupled to a quantizer scale input 
25 of quantizer 1 4 via quantizer scale filter 19. 

Figure 2 will be used to explain operation of the apparatus in terms of MPEG 
encoding. In a first step 21 during operation video input information is supplied to input 10. 
In the first step 21 pre-processor 12 pre-processes the video input information, splitting 
frames into blocks, performing a DCT (Digital Cosine Transform) of video information in 
30 each block. Pre-processor 12 supplies the resulting transform coefficients to quantizer 14. 

In a second step 22 quantizer scale selector 18 pre-selects quantizer scale 
values Qp, quantizer scale filter 19 filters these pre-selected quantizer scale values Qp 
temporally and/or temporally and supplies the filtered quantizer scale values Qf to quantizer 
14. 
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4 28.03.2002 
In a third step 23 quantizer 14 quantizes the coefficients received from pre- 
processor 12, Le. it replaces the received coefficients by nearby quantized coefficients from a 
limited set of quantized values and supplies the quantized coefficients to packager 16. This 
may be done for example by dividing the coefficients of a block (optionally minus an offset) 
by the quantizer scale value and taking the integer part of the quotient as quantized value. In 
a fourth step 24 packager 1 6 encodes and packages the quantized coefficients into an MPEG 
compatible signal and supplies that signal to output 17. The steps 21, 22, 23, 24 are 
periodically repeated as new frames come in. 

In second step 22 quantizer scale selector 18 pre-selects quantizer scale values 
Qp for respective blocks in respective frames of the incoming video information. Quantizer 
scale selector 18 supplies the pre-selected quantizer scale values Qp to quantizer 14 via 
quantizer scale filter 19 to control the step size between successive quantized values in the 
limited set of quantized values from which quantizer 14 selects the quantized coefficients. 

Any known method of selecting the quantizer scale values may be used to pre- 
select the quantizer scale values Qp. In one example quantizer scale selector 18 receives a 
bit-rate selection signal and pre-selects the quantizer scale values Qp so that on average the 
bit rate of the output signal at output 17 does not exceed the selected bit rate. The quantizer 
scale selector 18 computes ameasure of complexity for each block and allocates individual 
bit rates to each block in proportion to the complexity and so that the aggregate bit-rate of all 
blocks does not exceed the selected bit-rate. The quantizer scale value Qp of each block is 
then selected so as no to exceed the bit-rate allocated to that block. 

Quantizer scale selector 18 supplies the pre-selected quantizer scale values Qp 
to quantizer 14 via quantizer scale filter 1 9. Quantizer scale filter 1 9 performs a temporal 
and/or spatial low pass filter operation on the pre-selected quantizer scale values Qp and 
supplies the low pass filtered quantizer scale values Qf to quantizer 14. 

It will be appreciated that second step may involve a certain amount of 
lookahead. That is, pre-selected quantizer scale values may be computed for blocks of a 
frame before the blocks of preceding frames and/or preceding blocks in the same frame are 
quantized, so as to select the quantizer scale value Qf for those blocks of preceding frames 
and/or preceding blocks in the same frame. Thus, account may be taken of pre-selected 
quantizer scale values of future blocks. 

Figure 3 shows a graph 30 of pre-selected quantizer scale values Qf selected 
by quantizer scale selector 1 8 for blocks at the same pixel positions in successive frames of 
video information as a function of time (i.e. frame number). Figure 3 also shows a graph 32 
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of temporally filtered quantizer scale values Qf, as produced by quantizer scale filter. 19. It 
will be noted that the filtered quantizer scale values Qf 32 follow the general trend of the 
preselected quantizer scale values Qp, but that fluctuations around the trend are suppressed. 

Similar graphs may be drawn for the quantizer scales Qp and Qf as a function 
of the position of the blocks with which these scales are associated in a frame. 

Any method may be used to filter the pre-selected quantizer scale values Qp 
30. For example one might take the median of a number of successive pre-selected quantizer 
scale values Qp in a window of frames around the frame for which the filtered quantizer scale 
value Qf is selected. This removes fluctuations but allows sudden changes, for example when 
the content of the block changes abruptly. In another example filtering is realized by 
computing an average of successive pre-selected quantizer" scale values Qp in a window of 
frames around the frame for which the filtered quantizer scale value Qf is selected. The 
average may be a weighted average, which assigns more weight to pre-selected quantizer 
scale values Qp from frames close to the frame for which the filtered quantizer scale value Qf 
is selected. 

In another embodiment motion information is used to support filtering. In this 
case, filter 19 temporally filters the quantization values that are selected for blocks in 
different frames that are related to each other by motion vectors. That is, if it is estimated that 
the content of a first block in a first frame moves to a second block in a second frame and 
from there to a third block in a third frame and so on, temporal filtering is applied to the 
sequence of quantizer scale values formed by the quantizer scale values of the first block, the 
second block, the third block and so on and the filtered quantizer scale values are used to 
quantize the corresponding blocks. Again this reduces visible artifacts. The motion vectors 
may be obtained in any way, for example from the motion vectors used in MPEG encoding. 

It will be appreciated that, although the implementation shown in figure 1, 
using a quantizer scale value filter 19 is advantageously simple, other means for reducing 
fluctuations may be used. For example, a search may be used for a set of quantizer scale 
values that minimizes distortion without exceeding a predetermined bit-rate. In this case, the 
apparatus should use a modified measure of distortion that assigns an increased value to the 
distortion in case of fluctuations in the quantizer scale values. This may be done, for 
example, by using a measure of distortion based on the deviations between the original image 
and the encoded image, added to squares of differences between actually used quantizer scale 
value Qf at corresponding positions in adjacent frames and/or between adjacent positions in 
the same frame. 
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It will also be appreciated that, although the principles of reducing fluctuations 
in quantizer scale have been described, many modifications may be needed to account for 
peculiar details of the coding standard. For example, in case of the MPEG standard only a 
limited number of quantizer scale values are allowed. Therefore, the filtered quantizer scale 
values Qf should be selected from this limited number of values, but the preselected 
quantizer scale values Qp may be selected freely. Also the MPEG standard imposes some 
restrictions on differences between the quantizer scale values that may be used for different 
blocks in a macroblock. This can be realized for example by selecting filtered quantizer scale 
values Qf per macroblock. 

Furthermore, in the MPEG standard most changes in the quantizer scale value 
(e.g. changes between horizontally adjacent blocks) have to be signaled with additional 
signals, which are not needed when the quantizer scale does not change. The additional 
signals lead to increases in the number of bits in die compressed signal. Therefore, it may be 
desirable to weigh both the noticeable distortion due to changes of the quantizer scale and the 
1 5 increases in the number of bits in the compressed signal. 

The invention may be applied to compression that produces data for 
transmission or for storage of video information. The invention can be applied to data being 
stored to a data container device like a floppy disk a Digital Versatile Disc or a Super Audio 
CD, or a master or stamper for manufacturing DVDs or SACDs. The invention may also be 
10 applied to transcoding of compressed video information. 

Figure 4 shows a transcoding apparatus. The transcoding apparatus the 
apparatus contains a partial decoding unit 40, a requantizer 44, a packager 46, a quantizer 
scale selector 48 and a quantizer scale filter 49. The input of the transcoder is coupled to the 
output via, successively, partial decoding unit 40, requantizer 44 and packager 46. Quantizer 
:5 scale selector 48 has an input coupled to an output of the partial decoder and an output 
coupled to a quantizer scale input of requantizer 44 via quantizer scale filter 49. 

In operation the transcoder receives compressed encoded data with signal 
values that have been quantized using a first quantizer scale. The transcoder requantizes the 
signal values with a second quantizer, for example in order to increase the compression rate. 
Partial decoding unit 40 partially decodes a received compressed signal, and supplies 
information representing the quantized signal value from the received signal to the 
requantizer 44. Requantizer 44 selects a new quantized value for each signal value, so that 
the new quantized value belongs to a quantization scale indicated by filter 49. Packager 46 
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repackages the information from the stream into a new encoded stream, encoding the new 
quantized values. 

Selector 48 selects quantization scales. This is done for example for each 
block in proportion to the quantization scales used in corresponding block in the incoming 
5 signal (the proportionality factor being selected so that a measured bit rate at the output on 
average corresponds to an externally demanded bit rate). Filter 49 spatially and/or temporally 
filters the selected quantization scale values as described for figure 1 . 

Although separate hardware units have been shown in an implementation of 
the invention, it will be appreciated that the functions of one or more of these units may be 
10 combined in a single unit. For example, preselection and filtering, performed by units 18, 19, 
may be executed by a single unit. Furthermore, the components of the apparatus shown in 
figure 1 may be physically distributed over a number of apparatuses, while logically regarded 
as a single apparatus. Although the invention has been described in terms of compression 
according to the existing MPEG standard, it may of course be applied to any form of 
15 compression, for example according to future MPEG standards or nay other standard. 

Also, various elements may be implemented with a computer system 

programmed with suitable computer programs that implement the invention, the programs at 
least including instructions for performing steps of a method according to the invention when 
run on a computer system or enabling a general propose computer.system to perform 
20 functions of a computer system according to the invention. Such a computer program may be 
provided on a data carrier, such as a CD-rom or diskette, stored with data loadable in a 
memory of a computer system, the data representing the computer program. A data carrier 
may further be a data connection, such as a telephone cable or a wireless connection 
transmitting signals representing a computer program according to the invention. 

In the foregoing specification, the invention has been described with reference 
to specific examples of embodiments of me invention. The specifications and drawings are, 
accordingly, to be regarded in an illustrative rather than in a restrictive sense. 
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© 

A method of encoding a digital video stream, the method comprising the steps 
of 

- dividing frames of the video stream into blocks; 

- selecting quantizer scale values for respective ones of the blocks, under control of a 
complexity of a content of image information in the respective ones of the blocks, wherein an 
effect, of fluctuations of the complexity as a function of time and/or position of the respective 
ones of the blocks, on temporal and or spatial fluctuations of the quantizer scale values is 



1. 



- computing video data quantized according to the selected quantizer scale values; 

- generating an encoded video stream comprising the quantized video data. 



2. A method of encoding a digital video stream according to Claim 1, wherein 
the selecting step comprises 

- preselecting preliminary values for the quantizer scale values for respective ones of the 
1 5 blocks, under control of the complexity; 

- temporally and or spatially low-pass filtering the preliminary values to determine the 
quantizer scale values to be used in the computing step. 

3. A method of encoding a digital video stream according to Claim 1 , 
20 wherein the digital video stream is an encoded video stream comprising information 

representing quantized input values, and requantizing the quantized input values according to 
the selected quantizer scale values. 

4. A method of encoding a digital video stream according to Claim 1, the method 
25 comprising relating a succession of blocks from successive frames to each other on the basis 

of estimated motion vectors, the selecting step comprising temporally low-pass filtering a 
succession of quantization values that are provisionally selected for the successive blocks 
respectively. 
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5. A system for encoding a digital video stream, the system comprising 

- a pre-processor arranged to compute image information for respective blocks of pixels in 
frames of the video stream; 

- a quantizer arranged to quantize the computed image information, using block dependent 
quantizer scale values; 

- an encoded video stream generator arranged to generate an encoded stream that encodes the 
quantized video data; 



- a quantization scale selector arranged to select the quantizer scale values for respective 
of the blocks, under control of a complexity of a content of image information in the 
respective ones of the blocks, wherein an effect, of fluctuations of the complexity 
function of time and/or position of the respective ones of the blocks, on temporal and 
spatial fluctuations of the quantizer scale values is dampened. 



ones 



as a 
or 



6 - A system according to Claim 5, the system comprising 

15 -a preselector for preselecting preliminary values for the quantizer scale values for respective 
ones of the blocks, under control of the complexity; 

- a low pass filter unit for temporally and or spatially low-pass filtering the preliminary 
values to determine the quantizer scale values. 



20 7 - A s y stem according to Claim 5, the pre-processor being arranged to compute 

the image information from an incoming encoded video stream with quantized input signal 
values , the quantizer requantizing the quantized input values. 

8. A system according to Claim 6, the filter temporally low pass filtering 

25 preselected quantization that are pre-selected for a succession of selected blocks in different 
frames, the selected blocks being related to each other by motion vectors estimated for the 
frames. 



30 



9. A computer program product comprising a computer program with 

instructions for programming a computer to execute the method according to Claim 1 . 
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ABSTRACT: 2 8 Q3. 2QQ2 



During video compression quantizer scale values for respective blocks under 
control of a complexity of a content of image information. The temporal and or spatial 
fluctuations of the quantizer scale values are dampened, for example by low pass filtering 
provisionally selected quantizer scale values. The quantizer scale values are use to quantize 
the video data. The quantizer scale values may also be used during transcoding. 



Figs 1, 3 



PHNL020237 




